The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.

The automation of an increasingly large number of software engineering tasks is becoming possible thanks to Machine Learning (ML). One foundational building block in the application of ML to software artifacts is the representation of these artifacts (e.g., source code or executable code) into a form that is suitable for learning. Many studies have leveraged representation learning, delegating to ML itself the job of automatically devising suitable representations. Yet, in the context of Android problems, existing models are either limited to coarse-grained whole-app level (e.g., apk2vec) or conducted for one specific downstream task (e.g., smali2vec). Our work is part of a new line of research that investigates effective, task-agnostic, and fine-grained universal representations of bytecode to mitigate both of these two limitations. Such representations aim to capture information relevant to various low-level downstream tasks (e.g., at the class-level). We are inspired by the field of Natural Language Processing, where the problem of universal representation was addressed by building Universal Language Models, such as BERT, whose goal is to capture abstract semantic information about sentences, in a way that is reusable for a variety of tasks. We propose DexBERT, a BERT-like Language Model dedicated to representing chunks of DEX bytecode, the main binary format used in Android applications. We empirically assess whether DexBERT is able to model the DEX language and evaluate the suitability of our model in two distinct class-level software engineering tasks: Malicious Code Localization and Defect Prediction. We also experiment with strategies to deal with the problem of catering to apps having vastly different sizes, and we demonstrate one example of using our technique to investigate what information is relevant to a given task.

We propose LiDAL, a novel active learning method for 3D LiDAR semantic segmentation by exploiting inter-frame uncertainty among LiDAR frames. Our core idea is that a well-trained model should generate robust results irrespective of viewpoints for scene scanning and thus the inconsistencies in model predictions across frames provide a very reliable measure of uncertainty for active sample selection. To implement this uncertainty measure, we introduce new inter-frame divergence and entropy formulations, which serve as the metrics for active selection. Moreover, we demonstrate additional performance gains by predicting and incorporating pseudo-labels, which are also selected using the proposed inter-frame uncertainty measure. Experimental results validate the effectiveness of LiDAL: we achieve 95% of the performance of fully supervised learning with less than 5% of annotations on the SemanticKITTI and nuScenes datasets, outperforming state-of-the-art active learning methods. Code release: https://github.com/hzykent/LiDAL.

This paper proposes a hardware-efficient architecture, Linearized Convolution Network (LiCo-Net) for keyword spotting. It is optimized specifically for low-power processor units like microcontrollers. ML operators exhibit heterogeneous efficiency profiles on power-efficient hardware. Given the exact theoretical computation cost, int8 operators are more computation-effective than float operators, and linear layers are often more efficient than other layers. The proposed LiCo-Net is a dual-phase system that uses the efficient int8 linear operators at the inference phase and applies streaming convolutions at the training phase to maintain a high model capacity. The experimental results show that LiCo-Net outperforms single-value decomposition filter (SVDF) on hardware efficiency with on-par detection performance. Compared to SVDF, LiCo-Net reduces cycles by 40% on HiFi4 DSP.

Modality representation learning is an important problem for multimodal sentiment analysis (MSA), since the highly distinguishable representations can contribute to improving the analysis effect. Previous works of MSA have usually focused on multimodal fusion strategies, and the deep study of modal representation learning was given less attention. Recently, contrastive learning has been confirmed effective at endowing the learned representation with stronger discriminate ability. Inspired by this, we explore the improvement approaches of modality representation with contrastive learning in this study. To this end, we devise a three-stages framework with multi-view contrastive learning to refine representations for the specific objectives. At the first stage, for the improvement of unimodal representations, we employ the supervised contrastive learning to pull samples within the same class together while the other samples are pushed apart. At the second stage, a self-supervised contrastive learning is designed for the improvement of the distilled unimodal representations after cross-modal interaction. At last, we leverage again the supervised contrastive learning to enhance the fused multimodal representation. After all the contrast trainings, we next achieve the classification task based on frozen representations. We conduct experiments on three open datasets, and results show the advance of our model.

弱监督的点云语义分割方法需要1 \％或更少的标签，希望实现与完全监督的方法几乎相同的性能，这些方法最近引起了广泛的研究关注。该框架中的一个典型解决方案是使用自我训练或伪标记来从点云本身挖掘监督，但忽略了图像中的关键信息。实际上，在激光雷达场景中广泛存在相机，而这种互补信息对于3D应用似乎非常重要。在本文中，我们提出了一种用于3D分割的新型交叉模式弱监督的方法，并结合了来自未标记图像的互补信息。基本上，我们设计了一个配备有效标签策略的双分支网络，以最大程度地发挥标签的力量，并直接实现2D到3D知识转移。之后，我们以期望最大（EM）的视角建立了一个跨模式的自我训练框架，该框架在伪标签估计和更新参数之间进行了迭代。在M-Step中，我们提出了一个跨模式关联学习，通过增强3D点和2D超级像素之间的周期矛盾性，从图像中挖掘互补的监督。在E-Step中，伪标签的自我校准机制被得出过滤噪声标签，从而为网络提供了更准确的标签，以进行全面训练。广泛的实验结果表明，我们的方法甚至优于最先进的竞争对手，而少于1 \％的主动选择注释。

最近的隐私泄漏事件和更严格的政策法规要求公司和移动应用程序的合规标准更高。但是，此类义务还在应用程序开发人员遵守包含各种观点，活动和角色的这些法规方面面临重大挑战，尤其是对于在此问题或资源有限的小型公司和开发人员中。为了解决这些障碍，我们开发了一个自动工具NL2GDPR，该工具可以从开发人员的自然语言描述中制定策略，同时还可以确保该应用程序的功能符合通用数据保护法规（GDPR）。 NL2GDPR是通过利用由百度认知计算实验室开发的信息提取工具OIA（开放信息注释）开发的。核心，NL2GDPR是一个以隐私为中心的信息提取模型，附有GDPR策略查找器和策略生成器。我们进行一项全面的研究，以掌握提取以隐私为中心的信息和制定隐私政策的挑战，同时利用针对此特定任务的优化。借助NL2GDPR，我们可以在正确识别与个人数据存储，过程和共享类型相关的GDPR策略方面获得92.9％，95.2％和98.4％的精度。据我们所知，NL2GDPR是第一个允许开发人员自动生成GDPR策略的工具，只需要输入自然语言来描述应用程序功能。请注意，其他非GDPR相关功能可能与生成的功能集成在一起，以构建复杂的应用程序。

成功的基于机器学习的命名实体识别模型可能会因某些特殊领域的文本而失败，例如中文地址和电子商务标题，需要足够的背景知识。对于人类注释者来说，此类文本也很难。实际上，我们可以从具有一些共同实体的相关文本中获得一些潜在的有用信息，以帮助文本理解。然后，人们可以通过引用相关样本来轻松地提出正确的答案。在本文中，我们建议使用相关样品增强NER模型。我们通过大规模内域未标记的数据从稀疏的BM25检索器中绘制相关样品。为了明确模拟人类推理过程，我们执行了通过多数投票校准的无培训实体类型。为了捕获训练阶段的相关特征，我们建议通过基于变压器的多构度跨编码器对相关样品进行建模。上述两个域数据集的经验结果显示了我们方法的功效。

Federated学习（FL）最近作为一种增强隐私的工具而受到了极大的关注，可以由多个参与者共同培训机器学习模型。FL的先前工作主要研究了如何在模型培训期间保护标签隐私。但是，FL中的模型评估也可能导致私人标签信息的潜在泄漏。在这项工作中，我们提出了一种评估算法，该算法可以准确计算使用FL中的标签差异隐私（DP）时，可以准确计算广泛使用的AUC（曲线下）度量。通过广泛的实验，我们显示我们的算法可以计算与地面真相相比的准确AUC。

实体对齐是知识图融合中的至关重要任务。但是，大多数实体对准方法都有可伸缩性问题。最近的方法通过将大型公斤分成小块来解决这个问题，以嵌入和对齐学习。但是，这种分区和学习过程导致结构和对齐过度损失过多。因此，在这项工作中，我们提出了一种可扩展的基于GNN的实体对准方法，以从三个角度降低结构和对齐损失。首先，我们提出一种基于中心性的子图生成算法，以回顾一些具有不同子图之间桥梁的地标实体。其次，我们介绍了自我监督的实体重建，以从不完整的邻里子图中恢复实体表示形式，并设计了跨纸笔负面抽样，以在对齐学习中纳入其他子图中的实体。第三，在推理过程中，我们合并子图的嵌入，以制作一个单个空间进行对齐搜索。基准开放数据集和提议的大型DBPEDIA1M数据集的实验结果验证了我们方法的有效性。